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Abstract — This paper studies properties of entropy functions 
that are induced by groups and subgroups. We showed that 
many information theoretic properties of those group induced 
entropy functions also have corresponding group theoretic inter- 
pretations. Then we propose an extension method to find outer 
bound for these group induced entropy functions. 

I. Introduction 

The determination of capacity regions in communication 
systems (such as a channel or a network) is a classical 
information theory problem. Due to lack of tools, capacity 
regions are usually hard to determine except in some simple 
cases. Some examples of the available tools include Shannon 
and non-Shannon Information inequalities and LP bound. 

Take network multicast problem as an example. Consider 
data generated at multiple sources to be multicast to various 
receivers in the network. Each source is associated with a 
random variable denoting the random message generated at 
that source. Similarly, each edge/channel is associated with 
a random variable denoting the random message transmitted 
along the edge. Clearly, how these random variables relate to 
each other is not arbitrary but must depend on the transmission 
scheme and the underlying network topology. Specifically, 
they must meet the following criteria: (1) random variables 
associated with sources are independent to each other, (2) 
random variable with an outgoing edge of a node v must 
be a deterministic function of the set of random variables 
associated with sources can channels directing towards v, and 
(3) if the data generated at a source is destined to a node, 
then the random variable with that source must be a function 
of the random variables associated with incoming messages 
and sources available to that node. 

The capacity region for the multicast problem is essentially 
equivalent to determining the set of all entropy functions for 
sets of random variables meeting the above criteria. By using 
various information inequalities and LP bound, we obtain an 
outer bound for the capacity region by finding outer bounds 
for the set of entropy functions. 

If there is a restriction on the class of network codes (say, 
hnear network codes) that can be used, the set of entropy 
functions induced by the variables may meet additional re- 
quirements. For example, in [1], it was shown that if an 
abelian network codes (including linear network codes and 
R-module codes as special cases) are used, the set of random 
variables can then be "characterized" by abelian groups and 
consequently, its entropy function satisfies additional require- 
ment such as the Ingleton's inequality. Moreover, the capacity 



region for linear network codes can be determined similarly by 
identifying the set of entropy functions induced by the abelian 
groups. 

Unfortunately, the above characterization for the capacity 
region usually cannot be implemented in practice because the 
difficulty of determining the set of entropy functions meeting 
the required criteria. LP bound gives outer bounds for capacity 
region by replacing the set of entropy functions by the set of 
polymatroidal functions. Clearly, if one can tighten the bound 
for the set of entropy functions (or those induced by abelian 
groups), then one can get a tighter capacity region bound. 

The organization of the paper is as follows. In Section 2, 
we describe how groups and subgroups can induce random 
variables and demonstrate that many information theoretic 
properties of these group induced random variables have 
nice group theoretic interpretations. Section 3 focuses on the 
properties of those random variables induced by abelian groups 
including vector spaces. These abelian group induced random 
variables is of special interests because of its relation to linear 
network code capacity region. And finally in Section 4, we 
propose a method to outer bound the set of entropy functions 
by using a method called "extension". 

II. Groups, Random Variables and Entropies 

Random variables (and their corresponding entropy func- 
tions) induced/characterized by groups and subgroups have 
many nice properties. For example, [2] proved that an uncon- 
strained information inequality is valid if and only if it holds 
for all random variables induced by groups and subgroups. 
Through this relation, tools in group theory can be used to 
prove information inequalities, or vice versa. 

While [2] proved the equivalence of information theoretic 
and group theoretic inequalities, this paper focuses on the 
properties of entropy functions, especially those generated 
from abelian groups and subgroups. 

Let N — {1, • • • , n} and Xi, X2, • • • , Xn be n jointly 
distributed discrete random variables. For any nonempty subset 
a of N, Xa denotes the joint random variable {Xi : i E a) 
whose entropy is denoted by H{Xa)- Let JF„ be the set of all 
real functions defined on the collection of nonempty subsets 
of N. Hence, it is a (2" — 1) -dimensional real Euclidean space 
[7]. 

Definition 1. Let g G Tn- Then g is an entropy function (or 
is called entropic) if there exists a set of random variables 
Xi, X2, • • • , Xn such that g(a) = H{Xa) for any nonempty 
subset a 0/ N. 



Let r* be the set of all entropy functions and hence is a 

subset of Tn. It plays an important role in information theory; 

however, it also has a very complex structure. In [7], it was 
— * 

proved that r„, the closure of F*, is a closed and convex 
cone, and thus is much more manageable than F* . In fact, for 
many applications, it is sufficient to consider F„ instead of 
F* . However, except for n < 3, a complete characterization 
for the set F„ is still absent. 

The main result on the relations between groups, random 
variables and entropies is the following theorem proved in [2]. 

Theorem 1. Let G be a finite group and Gi,G2,--- , G„ 
be its subgroups. Then there exists a set of random vari- 
ables Xi , X2 , • • • , Xn ( said to be induced/characterized by 
(G,Gi,G2,-- - ,G„); such that H{X^) ^\og\G\/\f]^^^G,\ 
for any nonempty subset a of N. 

Proof: Suppose ?7 is a random variable having a uniform 
probability distribution over G. For each i g N, the subgroup 
Gi partitions the group G into log |G|/|Gi|'s left cosets of Gi 
in G. Then we define the random variable Xi as a function 
of U such that Xi is the index for the left coset of Gi in 
G that contains U . It was then shown in [2] that H{Xa) — 
log \ G\/\ HieQ ^'^y nonempty subset a of N. ■ 

Many nice information theoretic properties of random vari- 
ables induced by groups have group theoretic interpretations. 
Some examples are given as follows. 

Proposition 1 (Closed in addition). Let gi and g2 be group 
characterizable (i.e., gi and g2 are entropy functions for 
random variables induced by groups). Then gi + g2 is also 
group characterizable. 

Proposition 2 (Conditioning). Let g G Tn be group charac- 
terizable. For any non-empty subset (3 of N, define a function 
g^ G Tn such that g/3(a) = g(a U /3) — g(/3) for all subset a 
of N. Then g^ is group characterizable. 

Let (Xi, X2, • ■ • X„) be a set of n random variables 
and g be its entropy function. The proposition said that if 
(Xi, X2, • • • X„) is group characterizable, then the function 
g/3(a) = H{Xa\Xi3) for any non-empty subset a of N is also 
group characterizable. 

Proposition 3 (Functional dependency). Let (Xi, X2, • • • X„) 
be a set of random variables induced by a group characteri- 
zation (G, Gi, G2, • • • , G„). For any element i and subset a 
of N, we have H{Xi\Xj : j G a) — if and only if Clj^^ Gj 
is a subgroup of Gi. 

Remark: Proposition |3] is very useful in determining ca- 
pacity region for network codes. In [1], linear (and abelian) 
network codes are interpreted as group characterizations for 
the set of random message variables transmitted along the 
network links. By the proposition, the functional dependency 
between outgoing message and the incoming messages of a 
node can be rephrased as that the intersection of subgroups 
associated with incoming edges to the node is contained in 
the subgroup associated with the outgoing edge. 



Proposition 4 (Independence). Let Xi, - ■ ■ , Xn be in- 
duced by the group characterization (G, Gi, • • • , G„). Then 
Xi,-- - ,Xn are independent (i.e., H{Xi,--- ,X„) — 
EtiHiX.) if and only ;/nr=ilG.| - |G|"-i | H.^n I- 
In the case of n — 2, it is equivalent to that G — Gi o G2 
where o is the binary operator of the group G and Gi o G2 = 
{aob : a e Gi,b e G2}. 

III. Abelian and linear group characterizations 

As mentioned in the introduction, the determination of 
capacity region for linear network codes is directly related to 
determining the set of entropy functions induced by abelian 
groups( and vector spaces). We will first study properties of 
abelian group characterizable entropy functions. 

A. Abelian group characterications 

Definition 2. A group characterization (G, Gi, G2, • • • , G„) 

is called abelian if G is an abelian group. 

Random variables (and their entropy functions) that are 
induced by abelian group characterizations have many nice 
properties. 

Let g G Tn and /3 be a non-empty subset of N. Define 
A(g,/3) as a function in Ti such that for any nonempty 
subset a of N, A(g, /3)(a) — g(a) if a and (3 are disjoint, 
and that A(g,/3)(a) - g - E,e/3(g(N) - g(N\{j}))u, 
otherwise. Or equivalently, A(g,/3) = g — J^jepisi^) ^ 
g(N\{j}))uj where is the entropy function for random 
variables Xi, • • ■ ,X„ such that H{Xj) = 1 and iJ(Xj) = 
for all i j. 

Proposition 5. If g is abelian group characterizable, then 
A(g,/3) G Tn is also abelian group characterizable. 

In other words. Proposition |5] says that if random variables 
Xi , • • ■ , Xn can be characterized by subgroups of an abelian 
group, then , • • ■ , Xn can be "decomposed" into 2n's 
random variables Yi, • ■ • , Zi, ■ • • , Z„ such that 

1) X, = {Y,,Z,) for all i G N; 

2) H(Yn,Zn) = H(Yn) + Y.^e^H{Zi)■ 

3) H{Y,\Y^\i) = for all i G N. 

Therefore, each Xi is decomposed to two random variables Yi 
and Zi such that Zi is independent of all other random vari- 
ables, and all the "interaction" among Xi is captured through 
Yi. Consequently, to study the properties of {Xi, • • • , Xn), it 
is sufficient to study (Yi, • ■ • , y„) instead. 

Lemma 1. Let X and Y be two random variables. Then we 
can construct a random variable W such that 

1) H(W\X) = H{W\Y) = 

2) If Z is another random variables such that H{Z\X) = 
H{Z\Y) = 0, then H{Z\W) = 0. 

For notation simplicity, W will be denoted as W = X*Y. 

Remark: W — X*Y is unique in the sense that two random 
variables are said to be equal if one is a function of the other 
and vice versa. 



Let X,Y and Z be random varibles, then {X*Y)*Z ~ 
X*{Y*Z). and X*Y = Y*X. In other words, the operator 
"*" is associative and commutative. 

Roughly speaking, X*Y is the random variable with the 
maximal entropy such that it is a function of either X or 
Y alone. Note, that the entropy of X*Y is always upper 
bounded by the mutual information between X and Y. In 
fact, the bound is not tight in general. However, as will be 
demonstrated in the following proposition, the bound is always 
tight if {X, Y) has an abelian group characterization. 

Proposition 6. Let {G,Gi, ■ ■ ■ ,Gn) be an abelian group 
characterization for Xi, • • • , X„. Let X„+i — Xa*X0. Then 
(G, Gi, • • • , G„, G„+i) is an abelian group characterization 
for Xi,--- ,X„+i where G„+i = (fljGa G**) + (flje/j Gj). 
Furthermore, I{Xa] Xj3\Xn+i) = 0. 

B. Linear group characterizations 

Vector spaces are special cases of abelian groups and have 
many nice properties. For example, the logarithm of the size 
of any vector subspace is proportional to its dimension. In the 
following, we will show that if the group and its subgroups 
in a group characterization are vector space, then the induced 
entropy function can be defined in an alternative way by using 
orthogonal complements. 

Definition 3. Let Wi , W2 , • • • , Wn be vector subspaces of 
F™ where ¥ is a finite field. Then the group characterization 
(F™, M^i, M^2, • ■ ■ ,Wn) is called linear (with respect to the 
field ¥). 

It is easy to verify that if (F™, Wi, W2, • ■ ■ , W„) is a linear 
group characterization for an entropy function g, then 

IF" 

g(a) = log -J i— = (dimF™ - dim VK„) log |F| (1) 

Let x=(ai,a2,-- - ,a.m) and y=(6i,62,-'' , &m)- Define a 
hnear operator ( ) ; F" x F™ ^ F by (x,y) = 0162 + 
0262 + • • • + CLnibm- Then the vector x is called orthogonal to 
y (denoted as x _L y) if and only if (x, y) = 0. Let be a 
vector subspace of F™. Define = {x e F™ : (x, y) = 
Vy e VF}. Then is a vector space, and is called the 
orthogonal complement of W (with respect to F™). 

Tlieorem 2. Suppose g e J-n has a linear group characteri- 
zation (F™, VFi, T^2, • • • , Wn). Then 

g(a) = (dimF™ - AitclWo) log |F| = log |F| dim^W^-^. (2) 

In plain words. Theorem |2] said that g has a linear group 
characterization (F™, Wi, W2, • • • ,Wn) if and only if it is 
proportional to the rank function for the set of vector spaces 

The study of rank functions of vector spaces has a long 
history. The investigation on linear dependence (which can be 
determined by the rank functions) led to the development of 
matroid theory founded by Whitney in 1935. It turns out that 
matroid theory plays an important role in graph theory, pro- 
jective geometry, switching theory and other The importance 



of Theorem |2] is that it demonstrates the equivalence of linear 
group characterizable entropy functions and rank functions for 
vector subspaces, hence creates a link between group entropy 
functions and matroid theory. 

The following theorem is one important result that explic- 
itly demonstrates the difference between an arbitrary group- 
characterizable entropy function and one induced by by 
abelian groups. 

Tlieorem 3 (Ingleton's inequality). Let a function g G be 
abelian group characterizable. Then 

g(12)+g(13)+g(14) g(l)+g(2) + g(34) 

+g(23) + g(24) - +g(123) + g(124) 

A special case of Theorem[3]was first given in [4] when g is 
a rank function for vector subspaces. By Theorem|2l Ingleton's 
inequality must be satisfied by all entropy functions that have 
a linear group characterization. It was later proved in [3] that 
the inequality is also satisfied by abelian group characterizable 
entropy functions. Using a similar proof technique, we can 
show that Ingleton's inequality in fact is satisfied by a more 
general class of entropy functions what we call "pseudo- 
abelian group characterizable". 

Definition 4. An entropy function of a set of random variables 
{Xi, • • ■ , Xn) is called pseudo-abelian group characterizable 
if for any disjoint subsets a and (3 of N, 

1) {Xi : i Cz a) and {Xj : j E [3) are conditional 
independent given {Xi : i G a)*{Xj : j G (3); 

2) {Xi : i G a)* and {Xj : j G are conditional 
independent given {Xi : i G a)**{Xj : j G f3)* where 
{Xi : I G a)* is defined as Xi-^'^Xi^* ■ ■ ■ for all ii G a. 

Tfieorem 4. If entropy function g is pseudo-abelian group 
characterizable, then it satisfies the Ingleton 's inequality. 

This theorem is the first result showing that Ingleton's 
inequality has an information theoretic interpretation without 
resorting to the use of any group notations. 

IV. Bounding sets of entropy functions 
Definition 5. Define 

Tri {g G J-ji ■ g is group characterizable. } 

— {g G ■ g is abelian group characterizable. } 
TJj* — {g G J-n : g is linear group characterizable. } 

As mentioned in Section 1, information inequalities and LP 
bounds are useful tools to determine capacity region. Due 
to the difficulty in characterizing the above sets of group 
characterizable entropy functions, capacity region bounds are 
usually loose. In this section, we devise a systematic approach 
to obtain bounds for these group characterizable entropy 
functions, so as to tighten the capacity region bounds. 

Roughly speaking, the idea is by extension and projection. 
Suppose we want to obtain bounds for T„. We first show that 



any functions in T„ can be extended to a function in H Ti 
(with m > n) where TL is usually a subset in Tm and has a 
simple explicit characterization. Then any outer bound W for 
Tm gives an outer bound for T„ by projecting the set WnTY 
back to T„. It turns out that a loose outer bound for may 
lead to a tight outer bound for T„. 

A. Adhesive and Join extensions 

Theorem 5 (Adhesive extension). Let g G F* and a be a 
subset of N. Then there exists a function h G extending 
g SMc/i f/zflf 

1) /or any i G a, h(i, n + i) = h(i) = h(n + i) 

2) /or any nonempty subset (3 of N, h({i + n : i G /?}) = 
h({i : i G = g(/3). Hence, h w an extension of g; 

3) h({l,--- ,n}) + h{{n + l,---,2n}) 
h({l,-.. ,2n}) + h(a). 

Similarly, if g G TJ^**, then there exists a function h G 
extending g which satisfies the above three conditions. 

Proof: Let g be the entropy functions of Xi, ■ • • , X„. We 
define random variables X„+i, • • ■ , X2n such that (1) Xn+i — 
Xi for i G a, (2) the marginal distribution of Xn+i, ■ ■ ■ , X2n 
is the same as Xi,--- ,X„ and (3) {Xi,--- ,Xn) and 
{Xn+i, ■ ■ ■ ,X2n) are conditionally independent given {Xi : 
i G a). It can then be shown that the entropy function 
for Xi, - ■ ■ , X2n is the desired extension of g. Hence, the 
adhesive extension of g is closed in F*. A similar proof can 
also be found in [6]. 

The proof for that adhesive extension is closed in will 
be given in the appendix. ■ 

If the entropy function is abeUan group characterizable, the 
following proposition gives another way to extension. 

Proposition 7 (Join extension). Let g G T"''. For any 
nonempty disjoint subsets a, (3 of N, there exists a function 
h G T'^_^i extending g such that (1) h{n + 1,q;) — g(a) = 
h(a) and h{n + 1, /3) g(/3) = h(/3) and (2) h{n + 1, a) + 
h(n + l,/3) = h(n + l) + h(a,/3). 

Proof: A direct consequence of Proposition |6] ■ 

B. Extension by coding theorems 

In the previous section, extensions are made based on prop- 
erties of random variables. This subsection proposed another 
way to extend entropy functions by using many existing coding 
theorems. For example, the following theorem is based on the 
Slepian-Wolf distributed source coding theorem. 

Tlieorem 6 (SW Extension). Let g G F* be an entropy 
function for random variables {Xi , • ■ • , Xn). Then there exists 
a function h G extending g such that (1) h(n + 1, a) = 

g(a) = h{a), (2) h(n + 1) = g(a,/3) - g(/3) and that 
(3) h(n + 1,(3) = g{a, (3). Furthermore, / g G Tj^, then 
there exists h G '^n+i extending g and satisfying the three 
conditions. 

Proof: Consider two distributed sources represented by cor- 
related random variables Xa and Xp. Then one can construct 



a message variable Xn+i such that (1) Xn+i is a function 
of Xa, (2) the entropy of Xn+i is about H{Xa\Xp) and (3) 
Xa can be recovered from X„+i and Xp with small error 
probability. Using Fano's inequality, it impUes that there exists 
h G r„_|_]^ extending g and meeting the three requirements. 

Now suppose that g G Tj^' with a linear group characteriza- 
tion (F™, Wi, ■ ■ ■ , Wn)- Then for any nonempty subsets a, (3 
of N, there exists vector subspaces W„+i containing Wa such 
that (1) WaC^Wp = Wn+ir\Wi3 and (2) Wn+i + Wp = F". It 
is straightforward to verify that the entropy function induced 
by (F™, Wi, - ■■ ,Wn, Wn+i) is the desired extension. ■ 

Tlieorem 7. Let g G F* such that there exists disjoint subsets 
a, 13,-1 of H where g(a, 7) + g(7) = 7) + g(/3, l)- Then 
there exists a function h G F,j_|_2 extending g such that 

1) h(n +1,0;) = g(a) = h(Q;) and h(n + 2,7) = g(7) = 

h(7); 

2) h(n + 1) = g(a,7) - g(7) and h(n + 2) = g{[3) + 

g(7) - g(/3, 7).- 

3) h(n + 1, n + 2, a) = h(7i + l,n + 2). 

Proof: The extension proposed in this theorem is based on 
the distributive source coding problem with side-information. 
Consider two correlated sources represented by random vari- 
ables Xa and X^. Suppose Xp is another random variable 
such that Xa X~^ — > Xp forms a Markov chain. 

Then one can construct two message variable Xn+i such 
that (1) Xn+i is a function of Xa and X„+2 is a function 
of X^, (2) the entropy of Xn+i is about H{Xa\Xp) and the 
entropy of Xn+2 is about I{X^-, Xp), and (3) that Xa can be 
recovered from X„+i and Xn+2 with small error probability. 
Using Fano's inequality, it implies that there exists h G F„_|_]^ 
extending g and meeting the three requirements. ■ 

C. Bounding by extension 

We now begin with an example to illustrate the idea how 
to bound entropy functions by extension. 

Example 1. Let g be an abelian group characterizable entropy 
function for Xi and X2. By Proposition \6\ there exists a 
random variable X3 ^ Xi*X2 such that I(Xi; X2\X3) ~ 0. 
Let h be the entropy function for Xi, X2, X3. Then h is an 
extension of g such that 

h(l) = g(l) h(2) = g(2) h(l,2) = g(l,2) 
h(l, 3) - g(l) h(l) + h(2) - h(3) + h(l, 2) 

In other words, for any g G Tj'', we can extend g fo h G Tg** 
such that (1) h(l) + h(2) = h(3) + h(l, 2), h(l, 3) = h(l) 
and h(2,3) = h(2). Let 

( h(l) + h(2)-h(3)+h(l,2), ] 

n=lheJ'-^: h(l,3) = h(l) L 
[ h(2,3) = h(2) J 

Let Proj be the linear projection from J-3 to J-2. Then if W 
is an outer bound for then Proj(>V H gives an outer 
bound for T|''. 

More generally, we have the following proposition. 



Proposition 8. suppose that we want to bound the region Vn- 
Suppose also that for any g G V„ C it can be extended 
to h € Vm and that h is contained in a subset Ti.. 

Let Proj be the linear projection from Tm to Tn- If W is 
an outer bound for V,„, then Proj^^(>V fl 7i) gives an outer 
bound for V„. Furthermore, ifW and H. are polyhedral cone, 
then the obtained bound for V„ is also a polyhedral cone. 

In fact, the proposed method not only can find outer bound 
for entropy functions, it can also be used to find outer 
bound for entropy functions satisfying additional constraints 
(e.g., independency or functional dependency constraints). For 
example, based on the extension obtained in Theorem |7] one 
can obtain an outer bound for the set of entropy functions in 
r; such that g(a, 7) + g(7) = g(a, 7) + g(/3, 7). 

V. Conclusion 

In this paper, we consider random variables (and their 
corresponding entropy functions) that are induced by groups. 
Various properties for these entropy functions, especially those 
generated from abelian groups, are derived. By extending the 
entropy function to a "higher dimensional" ones, we obtain 
outer bounds for the set of entropy functions that are of 
interest. 

It can be shown that the new non-Shannon information in- 
equalities obtained in [8] and Ingleton's inequality for abelian 
groups induce entropy functions and can be obtained through 
the use of the method of extension. We believe that by making 
use of many source coding theorems in the literature, the 
method of extension can be used to derive new information 
inequalities and hence to tighten the bounds for a set of entropy 
functions of interest. As a result, a tighter capacity region for 
network codes can be obtained. 

Appendix: Proof of TheoremO 

Suppose the set of random variables (Xi, • • • ,X„) has a 
group characterization (G, Gi, • • • , G„). Let a be a nonempty 
subset of N. To prove Theorem |5l it suffices to prove the 
following stronger statement- if Ga is a normal subgroup 
of G, then there exists a group-characterizabtion for random 
variables (Yi, ■ • • , y„, Zi, • ■ • , Z„) such that 

1) for any nonempty subset /5 of N, H{Xi3) — H{Yp) — 

2) for any i £ a, H{Yi\Zi) = H{Zi%) = 0. Hence, 
{Yi, i £ a) — (Zi,i G a); 

3) Yn is conditional independent of Zn given 

To prove the statement, we first define the set K as {(a, b) G 
G X G : aGa — bGa}- By direct counting, the cardinaUty of 
K is equal to |G||Gq|. 

For any (a, 6),(c, d) G K, we define (a, 6) o (c, c?) as the 
element in (ac, bd). It can be verified easily that acK — bdK 
and hence the binary operation is closed in K. In fact, {K, o) 
is indeed a group - (1) the associativity of "o" follows from 
that G is a group, (2) the identity element is (e, e) and (3) the 
inverse of {a,b) G X is (a^^,b~^). 



Define a group homomorphism Ti : K — > G such that 
Ti(a, b) = a. Similarly, define T2{a, b) = b. For any subgroup 
L in G, we define subgroups and in K as follows: 



^ {{a,b)EK 

-2 A 



GL} = 



(3) 



L'^{{a,b)eK: beL} = T^-^L) (4) 



The subgroups and have many interesting properties 
including the following, 

PI: li^l/lL^I = \K\/\L^\ = \G\/\L\. (This property 
follows from that \K\ = \G\\Ga\ and \Li\ = IL2I = 
|^||G„|.) 



P2: for all j G a, G\ 



Gj {Or equivalently. 



T,-HG,) = T2-\G,)y, 

P3: For any subgroup H in G and i — 1,2, O GJ, = 

[HnGa^y (i.e., TrHH) nrrHG^) = T^-\H n 

Go,)). Hence, \H'r\Gi\ = |i?nG„||Ga|. 
P4: For any subgroups H and L in G, 

nGinL^ ^ {(a, b):a£Hr\Ga,beLn Ga}. 

Hence, nG^ nL^I = \H f]Ga\\Lf]Ga\. 
Now, we define subgroups Hi = Gj = Ti^^(Gi) and Li ^ 
Gf = T2-\G,) for z G N. Let (Yi,--- ,Y„,Zi,--- ,Z„) 
be random variables induced by the group characterization 
{K, Hi, - ■ ■ , Hn, Li, - ■ ■ , Ln). Then by property 3, for any 
nonempty subset f3 of N, H{Xf3) ^ H[Yp) = H{Zp). By 
property 2, for any z G a, H(Yi\Zi) = H{Zi\Yi) = 0. Hence, 
(Yi, z G a) = {Zi, i £ a). Finally, by properties 1 and 4, 



\K\ \K\ 
m\ \Lm\ 



\G\ \G\ 



\Gn\ 
\G\ 



\Gn\ 
\G\\Ga\ 



|Ga| \Gn\\Gn\ 
\K\ \K\ 



(5) 



(6) 



(7) 



Hence, Yn is conditional independent of Zn given Yq, and 
{K, Hi, ■ ■ ■ , Hn, Li, ■ ■ ■ , Ln) is the desired group character- 
ization. 
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